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ABSTRACT 

This paper presents a study that investigated 
academic achievement of students (total n»407) guided by student 
tutors compared with students guided by faculty tutors at the Law 
scnool of the University of Limburg, the Netherlands. Two methods of 
assessing students' achievement on end-of -course examinations were 
used: expert judgements and prepositional analysis. Academic 
achievement was measured using short-essay questions. Results of both 
methods used indicated that no differences in achievement occurred, 
students guided by student tutors perfonrod as well as students 
guided by staff tutors. Several explanations are proposed for these 
results. Among them are: (l) the possibility that the students guided 
by student tutors spend more time on self-study to compensate for 
their tutors' lack of expertise, or (2) that the staff tutors perhaps 
allow students less time and freedom for discussing the subjects in 
their tutorial groups. Contains 15 references and an appendix 
containing an example of an essay question for the course, "Unlawful 
Behavior." (GLR) 
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UNDERGRADUATE STUDENTS AS TUTORS: 
ARE THEY AS EFFECTIVE AS FACULTY IN 
CONDUCTING SMALL-GROUP TUTORL\LS? 



Jos C Moust & Henk G. Schmidt 



Department of Educational Research and Development 
University of Limburg 
Maastricht, the Netherlanck 



Abstract 

In the problem-based first year curriculum of the faculty of Law, small tutorial 
groups are guided by undergraduate student tutors or staff tutors. In this study, 
academic achievement of the students guided by student tutors is compared with 
students guided by staff tutors. Two methods of assessing students* achievement on 
end-of-course examinations are used: expert Judgements and prvpositional analysis. 
Results, of both methods used, indicate that no differences in achievement occur. 
Students guided by student tutors perform as well as students guided by staff tutors. 
Several explanations are proposed to explain these nsults. 
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Undergraduate students as tutors: are they as effective as faculty in conducting smali- 
group tutorials? 

Problem-based learning sceira to be an answer to the claim that learning dvauld be an 
active, self-directed process. One of the elements of problcm-ba^ learning is the 
tutorial group. The aims of small-group tutorial sessions are: To analyze problems 
offered by the faculty, to formulate their own learning goals and to synthesize some 
days later the information acquired by self study (Schmidt, 1983). In the tutorial group, 
students learn from, and teach, each other through the exchange of ideas, questioning 
and explaining the topics at hand. Tutorial groups arc guided by a tutor, usiwlly a 
faculty member. The tutw has several tasks. Primarily, the tutor facilitates the stutfents' 
learning process. He can do so through probing questions to challenge stixlents* 
knowledge deeply. He can help students to diagnose their own cognitive processes to 
become aware of their misunderstandings and misconceptions. He can bring into the 
discussion examples, metaphors and analogic to stimulate students to apply their 
knowledge. And he can cfcvelqj students' reasoning skills by giving them feedback 
about the way they proceed. Furthennore, the tutor stimulates the members of his 
tutorial group to analyze and synthesize the problems in procedural proper ways. In 
addition he has the iCi^ponsibiUty to help students keeping the learning process moving 
forward and co-operating in an appropriate way (Barrows, 1988). 

When the problem-based faculties of law, economics and the health sciences at the 
University of Limburg were confronted with large numbers of stucknts, the tesK^hing 
staff of these faculties fafxd the problem of insufficient numbers of staff tutors in 
relation to the number of tutorial groups. As an economic solution to this jxoblem. 



these faculties decided to involve advanced undergraduate students as tutors. Of course, 
using these undergraduate students as tutors immediately raised the questicm whether 
^udent tutor-guickd groups differ from staff tutor-guided grouF^ with respect to 
achievement* Staff tutors usually have mcMe subject-matter knowledge as compared to 
student tutors. Staff tutcHs^ expertise is more sophisticated timn those of advanced 
unckrgraduate students. As experts in their field, staff tutors would be better able to 
create a learning environment in which modification and recxganization of the cognitive 
structures of their students beccmies possible* Therefcne, they would in principle be 
more able to help their students to get a deeper understanding of the subjects to be 
studied than students guided by student tutors. Research on peer teaching in the c<mtext 
of problem-based learning, however, shows no differenres in achievcn»ent between stu- 
dents guickd by staff tutors or student tutors (De Grave, de Voider, Gijselaers & 
Danioiseaux, 1990; Gijsebers, Bouhuijs, Mulder & MuUink, 1987; De Voider, De Grave 
Sl Gijselaers, 1985). In all of th^e ^udies, achievement was as^ssed by true-false 
questions. It is questionable whether the a^essment was able to test deeper understand- 
ing of the subjects by the students. In one study in which students* acliicvcment was 
assessed by essay questions, students guided by staff tutors pcrfonncd significantly 
better tliat students guided by ^udent tutors (Moust, De Voider & Nuy, 1989). So, it 
may be p(^iblc that students guided by staff tutors benefit more of this guidance in 
cases where deeper insight in the subjects is ass^ed. 

In the present study, research is reported on academic achievement of students guided 
by staff tutors versus thc3se guided by student tutors. Academic achievement was 
measured using diort-essay questions^ Hie answers were analyzed udng two nwthods of 
sccving. First, the answers to the essays quc^ons were sccxed by content expcxts. 



Subsequently, a more elaborated analysis called "prepositional analysis* is conducted on 
a sample of the answers (Schmidt, 1982). 



Method 

Study 1 

The study was conducted in the I jaw School of the University of Umburg, the 
Netlierlands. 

Subjects . Subjects were all students attending small-group tutorials guided by student 
tutors or staff tutors in two courses of the first-year law cuiriculum. In the first course 
230 students participated, guided by seven staff and six student tutors. In the second 
course, 177 freshman engaged in 18 groups guided by four staff and five student tutors. 
Each tutor guided two small-group tutorials. Students and tutors were randomly assigned 
to the tutorial groups. Each course lasted for eight weeks. Students met in their groups 
twice a week for two hours. Student tutois were rccmited by an C3pen selection pro- 
cedure. Seven third- and fourth-year undergraduate students were selected to fdlfill the - 
role of tutor during tlic first curriculum year. 

Materials. At the end of both courses all students had to take an achievement test. 
These tests consisted of a set of five open questions of tlie essay format, from which 
each student had to select three questiones to answer. The te^ was cksigi^ to measure 
more than fiac^l knowledge. By requiring use of law concepts and rules, and by 
stressing tlwt full ai^umentation ra the answer was necessaty, students wore ^uxmraged 
to give answers showing insight (see appendix 1 for an example) 



Procedure. After the examination the written anonymous answers made were randomly 
distributed among seven staff members with sufficient content expertise. Every judge 
independently scored, with the help of a global model answer and a scoring key, the 
student answers. The answers were scored from 1 (very bad) lo 10 (very good). The 
course grade was based on the mean score on the three answers. The data were 
analyzed using c«ie-way analysis of variance (ANOVA). 

Results 

Table 1 shows the results of the end-of-coursc tests for both courts 1 and 2. There 
were no significant differences in achievement between ^udents guided by :^udent tutors 
and students guided by staff tutors, llie results of the average mean score in the first 
course are F (1, 228) = 0.002, p < .96, the results of the successsive course are F (1, 
175) = 1.42, p < .23. 

insert Table 1 about here 

The question whether students guided by staff tutors achieve belter learning results than 
stu(knts guided by student tutors has to be answered in the negative. Student tutors 
seem to be able to facilitate students' learning as well as staff tutors. However, the 
answers of each sepcrate question were judged by only one content expert. Reliability 
^imates of the m^urement {nocedure could noi be made. So, the ^sesanent of 
students' answers could be rather objective (Coffman, 1971). Lack of reliability of 
measurement could effi^t the scores of individuals and, hence, the averages found in 
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the present experiment. So, it was decided to score a sample of tlie student's answers in 
a more objective way, that is by propositiomil analysis (Schmidt, 1982; Schmidt, Dc 
Voider, Dc Grave, Moust & Patcl. 1989). 

Study 2 

Propoditional analysis is a method used for text analysis (Meyer, 1975). PrqxMations are 
subject-predicate units, each expressing one single idea. The boundaries of two adjacent 
propositions can be found by identifying appropriate linguistic markers in the text: con- 
junctions, adverbs, relative pronouns, semicolons, and full stof» (Schmidt, et al,, 1989). 
An example is provided by the following protrxiol (slashes mark boundaries): 

Mrs. Custcrs agrees with Jongen Inc.(l)// that Jongen Inc. will do up her hou^ 
in excliange for a payment in money by the Custers* family, (2)// in other words 
a legal relationship is established between the Costers' family and Jongen Inc. 
(3)// Jongen Inc. sends Richard van Daien to the Custers' house. (4)// Richard is 
employed by Jongen Inc. (5)11 Jongen Inc. makes use of the help of other p«^Ic 
for the fulfillment of its legal obligation towards the C^isters' family (6)// Ait. 
6.1.8.3. NBW holds Jongen Inc. liable for Richard* behavior,(7)// in the same 
way as for its own. (8)//. Tlie Custers' family can sue Richard on the basis of 
art. 6.3.3.1. NBW (9)// 

Subjects. From the material of the first course 43 answers from each condition and 
from the second course 74 answers from each condition were randomly selected- A 
selection was made since the analysis of written prc^ocols is quite laborious. 
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Procethire. In the scoring process, llie answers produced by the subjects were first 
parsed into propositions by two intkpendcnt judges, llie interrater agreen^nt for this 
task was 94% Next, each proposition was checked for accuracy. Inaccurate propositions 
were removed. In addition, a distinction was made between numbers of explanatiMy and 
descriptive propositions produced in the accurate answers (Bromage and Mayer, 1981). 
An explanatory propcsition was dcfmed as a statement that is characterized by a 
juridicial qualification (f.e. the propositions #3. #7. #8 and # 9). All other cofre<a 
propositions were considered ctescriptive (f.e. the propositions # 1, #2, #4, # 5 and # 6). 
Acconfing to Mayer (1985), the number of explanatory proportions in free recall is a 
sensitive measure of depth of integration of subject matter into existing knowledge 
structures. Interrater agreement for this task was 81%. Differences of opinion between 
judges were solved by discussion. 

Results 

The results of the prop<»itional analysis: the number of accurate, descriptive and 
explanatory propositions arc shown in Table 2 and 3. Differences between means were 
tested by one-way analysis of variance (ANOVA), 

insert Tables 2 aiul 3 about licit: 

The results suggest that the answers of the first course show no difTerences between 
both conditions neither for the total jiumber of accurate propositions (F (1, 84) = .06, p 
< .81), nor for the descriptive propositions (F (1, 84) •= 1.66, p < .20) and the explana- 



toiy propositions ( F (1, 84) «= 0.03. p < .87). ITic results of the proposition analysis in 
the successive couisc show a similiar pattern. Number of accurate propositions is (F 1, 
146) = .01, p < .93), number of descriptive propositions is (F (1, 146) ■= 3.50, p < .06) 
and number of explanatory pn^itions is F (I, 146) = .39, p < .54). If we agree with 
Mayer (1985) that the number of cxplanatoiy propositions suggest the depth of in- 
tegration of the subject matter, than the question whether students guided by staff tutors 
achieve better Icaming results than students guided by student tutors must be answered 
in the negative. 

General DiscusMon 

In this paper an experiment was described in which staff members as tutors were 
compiled to third- and fourth- year undergraduate law ^udents as tutors in two ccnirses 
of the flist curriculum year of the law faculty. Student's achievement was assessed by 
essay questions at the end of the course. The itssults suggest that students guided by 
staff tutors did not perform better than «^udenls guided by student tutors. A more 
elaborated analysis of a sample of student's answers, through propositional analysis, also 
indicated that there were no significant differences. These results indicate that student- 
tutors ai« able to fulfill the tutor role as well as staff members. Several explanations are 
possible for this finding. One possible explanation would be that the students guided by 
student tutors spend more time on self study to compensate their tutors lack of 
expcttise. Our own research suggest that this may be the case, at least paitially. Tlic 
same group of students filled out weekly a study lime rcgisttation fcam. On this fomi 
students keep track of their time-oy-task. Results of this research indicate that in the 
first course students guided by ^udent-tutors spend significantly more time on self study 



tlian students guided by staff tutors, hi the second ccHirse there were no difTercnccs, 
Another explanation would be tlwt staff tutors pertiaps allow students less lime and 
freedom for discussing tlie subjccLs in their tutorial groups. Bender (1983) and Webb 
(1983) found that stall teaclieis took 46 to 61 % of the discussion time of the groups 
ihcy guided. In an instructional setting where there is more equality and mutuality 
between students and tutor, students may feel more free to express opinions, ask 
qu^ions and speculate about the problcm-at-liand. When staff tutors take the flocff most 
of the time, students are not. able to formulate their own Ickas, to question and exj^in 
subject-matters to each other. A third possible explanation would be that ^udent tutors 
behave in another way than staff tutors. Research by Moust (in press) indicates differen- 
ces in the way staff and student tutors handled their expertise, their authority and the 
interpersonal relationship to the members of tlie tutorial groups. 
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Appendix I. An example of an essay question, course "Unlawful bchavfor." 

Rkhaid van Dalcn is working as a house-paintcr» employed by Johnsons* Painting Inc. 
He is working for several weeks at the estate of the Clusters' family. ITic house has to 
be painted thoroughly, from the inside and the outside. During the paintworks the Ous- 
ters' family is on holidays in Switzerland. After several days Richard discovers that the 
Custers* femily is a glassware buff. The family has a valuabk collection displayed in 
scMne showca«js in the living room. 

By coincickncc Richaitte'cousin Alfred is also very interested in antique glassware. To 
do his cousin a favor, one evening Richard sugg^ to go to the Custers* house and 
take a look, Couan Alfred is very pleased with this offer. Since Richard has a key of 
the Civets* house, they get inside without trembles. After they have acbmred the 
antique glassware for a while, an accident happens. Richaid slips and falls, by unknown 
causes, with all his weight against one of the showcases. The showcase with all the 
antique glassware smashes on the floor in a thousand pieces. 

a. Indicate whether the Custers' family can claim for damages by Richard van 
Dalen or Johnsons' Painting Inc.? 

Give a mcrtivatcd answer. 

b. Indicate who has to j^y the damage in the end? 
Give a motivated answer. 
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Table L Means and standard deviations of the students' total test scores in the the first 
and sectfnd course of the first year curriculum of the faculty of law. 





course 


1 




course 


2 




Experimental 








N 


M 


Sd 


condition 


N 


M 


Sd 


Students gui(^ 










5.2 


1.7 


by staff tutors 


125 


5.6 


1.6 


82 


Students guicied 








95 


4.9 


1.6 


by student tutors 


105 


5.5 


1.5 


Total 


230 


5.5 


1.6 


177 


5.1 


1.6 
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TaUe 2 Average number of Accurate, Descriptive and Explatuitory Propositions in the 
first course. 



Accurate prop. Descriptive prop. Explanatory prop. 



Group M Sd M Sd M Sd 



Students guickd 
by staff tutors 

(n- 43) 24.6 13 ^ 3.7 3.3 21.0 11.0 

Students guided 
by student tutors 

(n = 43) 25.3 12.6 4.7 4.2 20.6 10.3 



Total (n = 86) 25.0 12.6 4.2 3.8 



20.8 10.6 
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Table 3 Average number of Accurate, descriptive and Explanatory Propihsitions in the 
second course. 



Accurate prop. Descrii^ivc prop. lixplanatory prop. 



Group M Sd M Sd M Sd 



Stucfents guided 
by staff tutors 

(n = 74) 21.1 10.1 4.0 4.3 17.1 9.6 

Students guided 
by student tutors 

(n « 74) 20.9 10.1 2.9 3.1 18.0 9.4 



Total (n - 148) 21.0 10.1 3.5 3.7 17.5 9.5 
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